International Review of Research in Open and Distributed Learning 
Volume 17, Number 4 


June-2016 

The Effect of Multilingual Facilitation on Active 
Participation in MOOCs 



Jean-Frangois Colas 1 , Peter B. Sloep 2 , and Muriel Garreta-Domingo 3 

1 ’ 3 Open University of Catatonia, Barcelona, Spain, 2 Weiten Institute, Open University of the Netherlands 


Abstract 

A new approach for overcoming the language and culture barriers to participation in Massive Open 
Online Courses (MOOCs) is reported. It is hypothesised that the juxtaposition of English as the language 
of instruction, used for interacting with course materials, and one’s preferred language as the language of 
participation, used for interaction with peers and facilitators, is preferable to “English only” for 
participation in a MOOC. The Hands-On ICT (HANDSON) MOOC included seven teams of facilitators, 
each catering for a different language community. Facilitators were responsible for promoting active 
participation and peer tutoring. Comparing language groups revealed a series of predictors of intention to 
learn, some of which became apparent in the first days of the MOOC already. The comparison also 
uncovered four critical factors that influence participation: facilitation, language of participation, group 
size, and a pre-existing sense of community. Especially crucial was reaching a sufficient number of active 
participants during the first week. We conclude that multilingual facilitation activates participation in 
MOOCs in various ways, and that synergy between the four aforementioned factors is critical for the 
formation of the learning network that supports a social dynamic of active participation. Our approach 
suggests future targets for the development of the multilingual and community potential of MOOCs. 

Keywords: MOOC, multilinguality, facilitation, language barrier, language of participation, cultural 
translation, peer tutoring, community, Dunbar’s number, group size, learning network, connectivist 
dynamics, active participation, inactivity, intention to learn 
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Introduction 


MOOCs and Multilingual Education 

Massive Open Online Courses (MOOCs) have become popular instruments for opening up education to 
non-traditional student audiences, including those from different cultures and language groups (Deimann 
& Vogt, 2015; Fitzgerald, Wu, & Witten, 2014). MOOCs are portrayed as being widely accessible. 
However, English is their predominant language, so one seems to assume that non-English speakers 
would easily be able to understand and digest English language content. However, research shows that a 
mere 20% of the world's population is able to profit from English only educational content (Beaven, 
Comas-Quinn, Hauck, de los Arcos, & Lewis, 2013). Thus, it is fair to assume that many online learners 
will stumble upon a language barrier in their study of MOOCS if these are offered in English only. 

MOOCs, therefore, would do well to become multilingual. This is particularly so for Europe (De Rosa, 
2014). There, MOOCs in different languages would promote cross-cultural and multilingual learning, 
helping to preserve Europe’s rich cultural, educational and linguistic heritage (Brouns, Serrano Martinez- 
Santos, Civera, Kalz, & Juan, 2015; Lundahl, 2014; Sloep & Schuwer, 2015). Furthermore, leveraging 
Europe’s linguistic richness (24 official languages) creates an enormous opportunity for reaching target 
audiences in other parts of the world as well. Yet, even though Europe recognises it is the moment to latch 
onto the opportunities offered by MOOCs (e.g., Deimann & Vogt, 2015), European Union (EU) MOOC 
activities are mainly concentrated in Western and Southern Europe, do not fully reflect the cultural 
diversity of Europe, and serve a limited number of language communities only (Porto Declaration on 
European MOOCs, 2014). Even though the majority of Europeans agree that all 24 languages spoken in 
EU should be treated on a par, English is the language that Europeans are most likely to be able to speak 
as a second language (Perifanou, Holotescu, Andone, & Grosseck, 2014). In actual fact, then, 
multilinguality still is a distant ideal. 

Perhaps another approach is needed, one in which English plays the part of a lingua franca, that 
facilitates the translation of courses into any other European language. Initiatives such as the European 
Multiple MOOC Aggregator (EMMA; http://proiect.europeanmoocs.eu/proiect/) capitalise on this idea 
by providing multilingual access to European MOOCs, thus bringing within reach the ideal of a pan- 
European, multi-linguistic, cross-cultural approach to online learning that reaches students beyond 
national boundaries (De Rosa, 2014; Brouns et al., 2015). MOOCs on EMMA are the first to be offered in 
three languages: Spanish, English and Italian. EMMA translates video subtitles and text content with the 
help of advanced speech recognition and machine translation technology, thus providing access to any 
MOOC’s learning materials in three languages. The next step would be to expand the number of target 
languages. Although not foreseen, this is doable. Taking yet one step further, EMMA would also 
automatically translate interactive content, say from the native speaker's tongue into English via a 
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translation service. At present, however, this is one step too far as the quality of the translation leaves too 
much to be desired. 

And yet, even if automatic translation would be of a high quality, it is a misunderstanding to believe that 
MOOCs can be made accessible to everyone just by translating them. Translation alone can’t fully replace 
course offerings developed by people who have a deep understanding of the cultural context in which 
students learn. Contextualisation, or cultural translation, makes MOOCs relevant to students in their 
respective cultural settings (Nkuyubwatsi, 2014). As the proponents of EMMA judiciously notice, offering 
MOOCs in multiple languages evokes a new challenge, that of the necessity of tutors to cover the entire 
course language spectrum (Civera, 2015). 

So although translation no doubt increases MOOC accessibility, it still leads to a loss of pedagogical 
quality. Besides, most likely, cultural translation cannot be automated in the foreseeable future, if at all. 
We therefore sought to address this issue in a novel way. It is grounded in the distinction between the 
passive and active command of a language: people may well be able to read a foreign language, yet find it 
difficult to interact with other people in it. Therefore, for a particular MOOC, we decided to offer to people 
from different language groups English-only written content. However, we tackled the issue of cultural 
diversity that direct translation fails properly to address by offering multilingual facilitation, i.e. 
facilitation in one’s own preferred (native or otherwise preferred) language. 

Facilitation of Participation 

Within the virtual learning environment of a MOOC teachers clearly play a different role than in ordinary, 
face-to-face teaching environments. Indeed, to help overcome barriers to participation in MOOCs new 
teacher roles have emerged that vary depending on the kind of MOOC. “Teachers” in the connectivist 
MOOCs, or cMOOCs, express educational goals differently than teachers in the institutionally rooted, 
highly structured extensions of MOOCs known as xMOOCs: they facilitate self-directed learning rather 
than provide access to expert knowledge (Ross, Sinclair, Knox, Bayne, & Macleod, 2014). Facilitators need 
to guide the learners and their communities to get involved and embrace social media practices (Kop, 
Fournier, & Mak, 2011; Medina-Salguero & Aguaded-Gomez, 2013). Importantly, facilitation also needs to 
address differences in cultural communication patterns. For learners who are not fully acculturated in the 
course’s cultural backdrop, such differences easily become barriers to participation and learning, and even 
lead to isolation. This is particularly likely to happen when learners do not know each other well (i), the 
communication medium is impoverished and contains reduced non-verbal cues only (ii), and meaning is 
mostly carried by asynchronous, written means of communication (iii) (Rovai, 2007). 

This view of facilitation is also espoused by De Waard et al. (2014). They notice that, to encourage 
multilingual participation in a MOOC, one should create small, open learning groups based in the various 
learners’ native languages or in languages they feel comfortable with. Such “local” open learning groups 
can offer in-person networking and assistance (Godwin-Jones, 2014). They also provide an environment 

in which peer learners can tutor each other. The groups then effectively work as peer learning 
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communities, allowing the burden of providing feedback to shift at least in part from the facilitator to the 
learners themselves (De Vries et al., 2005). Peer learning communities are justified in terms of 
“sustainability plans for the instructors” that obviously drastically change the teacher role (Ross et al., 
2014). 

Research shows that such online peer learning communities - sometimes called ad-hoc transient 
communities - can indeed be effective, efficient and attractive learning aids (Fetter et al., 2012; Sloep, 
2009). However, peer tutoring can pose severe problems to many learners. First, they may not have the 
necessary skills to act as tutors (Hsiao, Brouns, Bruggen, & Sloep, 2015). Second and of particular 
relevance in the present context, they may not be able to bridge cultural divides. This goes of course 
especially for many non-native English speakers who participate in MOOCs offered in English (even when 
facilitation is offered in the learner’s own language). As argued such MOOCs are inevitably developed 
from the developer’s own cultural perspective. Participants in such MOOCs cannot but align themselves 
with that cultural perspective. Although not impossible, it is obviously hard then to bridge cultural divides 
on one’s own. To be able effectively to do so, a facilitator is needed who is conversant with the MOOC’s 
cultural backdrop and that of the open learning groups in it. 

The HANDSON MOOC experiment 

The facilitation approach sketched in the above we investigated in the Hands-On ICT (HANDSON) MOOC 
(http://www.handsonict.eu) a course offered to educators at all educational levels aimed at letting them 
become more conversant with the use of Information and Communications Technology (ICT) in their 
teaching. To the best of our knowledge, it is the first MOOC to incorporate various (seven) teams of 
facilitators, each responsible for promoting participation and peer tutoring in a different language 
community. Our approach was suggested by exploratory research that the HANDSON project team 
carried out during an earlier pilot of the course (in which 73% of the participants originated from Greece; 
Garreta Domingo, Hernandez-Leo, Mor, and Sloep 2015b). Our findings indicated that, while most 
participants (themselves educators) were able to read the learning materials in English, they were 
reluctant to join the English spoken, live videoconference sessions (Google Hangouts). Crucially, the 
Greek forum that was offered as an impromptu solution, proved to be used intensely throughout the 
entire pilot. 

Completion rate for this earlier pilot of the MOOC was 9.2%, already beating the 5% rates thus far 
observed in MOOCs (Jordan, 2014; http://www.katyjordan.com/MOOCproject.html) and those obtained 
in similar MOOCs for teacher training such as the Open Learning Design Studio's MOOC ( OLDSMOOC , 
http://www.olds.ac.uk/: 1.2%) or the ICT in Primary Education MOOC 

(https://www.coursera.org/course/ictinprimarv : 5.3%). Also, similar to the trend identified by Jordan 
(2015), the first two weeks of the earlier pilot appeared to be critical in gaining student engagement. These 
positive preliminary results led us to further investigate facilitation in the student’s native language. 
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The HANDSON MOOC pilot reported here guided participants through the various steps of a design 
process, with the aim of teaching them to produce design artefacts (Garreta Domingo, Hernandez-Leo, 
Mor, & Sloep, 2015a). This set-up demanded learners each to develop their own learning project, which 
was to be tied to their own specific needs and cultural contexts. They were also stimulated to do so co¬ 
operatively. Although the learning materials were still provided in English only, the learners could 
produce artefacts and interact in forums with peers and facilitators in the language of their choice, usually 
their native tongue. We refer to the language in which the content was developed and made available to 
the learners as the language of instruction. The language in which the learners produced their artefacts, 
interacted with each other and with their facilitators we call the language of participation, to emphasise 
that only through active linguistic interaction genuine participation is achieved. Seven languages of 
participation were offered to the learners: Bulgarian, Catalan, English, French, Greek, Slovenian or 
Spanish. 

We began by suggesting that multilinguality of MOOCs would address issues with their accessibility. We 
argued that, although efforts to translate MOOCs have been made, simple translation does not address 
existing broad cultural differences and that multilingual facilitation is called for. This then is why we 
developed the approach just described. It involves two research hypotheses. The first addresses the issue 
of whether the use of English as a language of instruction for anyone, regardless their native language, 
does not pose an unsurmountable barrier. If it would do so, we would consider our approach as failed: 

Hi The use of English as the language of instruction, even for non-native English speakers, does 
not stand in the way of MOOC access. 

The second hypothesis is our main hypothesis: it addresses the question of whether multilingual 
facilitation indeed fosters productive MOOC engagement. Productive MOOC engagement should be 
evident not only from the participants’ appreciation but predominantly from increased completion rates: 

H 2 The use of the various participants’ preferred languages as the languages of participation 
increases the success of MOOC engagement. 

Research Method 

To tackle the complexity of the language and culture barriers to participation in a MOOC, the HANDSON 
project used a design-based research approach. It allowed us to study in naturalistic settings the impact of 
the incremental changes we administered over three pilots. The pilot reported here is the third in 
succession. Its focus is the evaluation of the effects of creating seven language groups to support 
multilingual facilitation. Information on the other pilots may be found in Stoyanov, Sloep, De Bie, and 
Hermans (2014) for pilot 1 and in Garreta Domingo, Hernandez-Leo, Mor, and Sloep (2015b) for pilot 2. 

This third pilot ran for five weeks, starting October 24, 2014. It was extensively advertised through many 
different sources and communities as the Learning Design Studio for ICT-based Learning Activities 
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MOOC, because it made extensive use of the Learning Design Studio approach by Mor and Mogilevsky 
(2013). 

The MOOC was implemented in the Canvas Virtual Learning Environment (VLE; www.canvasvle.co.uk/ 1 . 
The activities and (English) resources were made available to all participants as Canvas Modules. Each 
weekly module corresponded to one of the five phases of the Learning Design methodology: initiate (week 
1), investigate (week 2), inspire & ideate (week 3), prototype (week 4) and evaluate & reflect (week 5) 
(Garreta Domingo et al., 2015a; Stoyanov, Sloep, De Bie, & Hermans, 2014). For the course to be 
facilitated in seven languages, seven parallel learning groups were created, each one linked to a separate 
Canvas group that functioned as the group’s discussion space. Each language-specific learning group met 
every week for discussion in a live convergence session. 

Each of the seven language groups was served by its own facilitator. Volunteer facilitators with an 
adequate command of English and of at least one of the seven different languages used were recruited 
through a call on the HANDSON website and the HANDSON consortium members’ networks. They were 
trained through a series of informal video-conference meetings with the organisers. These sessions were 
focused on the Learning Design methodology, the MOOC activities, and the MOOC ICT Tools (Canvas, 
ILDE, Hangouts) (see “Table Pre-Course Activities - Facilitators Training” in the Appendix). In total there 
were 16 active facilitators: four were Bulgarian, two Catalan, three English, two French, three Greek, and 
two Spanish; due to the low activity in the small Slovenian group, after the first week facilitation was 
made available to its members through the English group only. 

Participant experience was evaluated at the end of the course through questionnaires about the course 
and its approach, specifically on facilitation and multilinguality (see Appendix). Data were collected by an 
e-survey, using the Lime Survey web application. Closed questions were used, based on a 5-degree Likert 
scale (1 representing the lowest grade and 5 the highest). Data were analysed using descriptive statistics. 
Eighty-two fully-filled out responses to the survey were received from the participants. The language 
group distribution of these 82 respondents is given in ‘Table PostPilot3 - Which language group did you 
join” in the Appendix. As the survey was filled in anonymously, a break-down of responses by language 
group is not available. 

Description of Participants 

In total 1691 people joined the pilot by creating a username in the Canvas environment of the course 
(their age and gender was not recorded). Of them, 902 joined one of the seven language groups and were 
active during the first week at least (Figure 1). Note that being part of a language group was needed to 
participate in any of the MOOC activities. Although the English and Spanish groups were the biggest ones 
by a small margin, the 902 participants who joined some language group were distributed fairly evenly 
over the groups. The only exception was the Slovenian group, counting 19 members only. Excluding the 
Slovenian group, which was short-lived, the average group size was 147.2 participants. 
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Participants registered (with Canvas username) 

1691 


Did not join any language group 

789 


Joined a specific language group 

902 

100% 

Bulgarian group 



128 

14.2% 

Catalan group 



140 

15.5% 

English group 



210 

23.3% 

French group 



121 

13.4% 

Greek group 



107 

11.9% 

Slovenian group 



19 

2.1% 

Spanish group 



177 

19.6% 



60 120 180 240 




Figure l . Distribution of participants in the seven language groups of the HANDSON MOOC. 

For four reasons, the set-up deliberately did not include a control group of learners, who would have had 
to be tutored in a language different than their own. First, methodologically speaking, the entire 
HANDSON study was set up as a design experiment, with incremental changes administered over three 
pilots (Burkhardt & Schoenfeld, 2003). The first pilot’s set-up was suggested by a literature survey 
(Stoyanov et al., 2014), and changes were subsequently implemented in pilots 2 and 3 as a result of 
experiences in pilots 1 and 2 respectively (Garreta Domingo et al., 2015b). Such a design study does not 
lend itself naturally to randomised experimental designs with control groups. Second, one of the ideas 
driving the HANDSON project was to beat the usual high dropout rates (cf. Hypothesis 2). Those rates 
have been reported widely (Jordan, 2014, 2015), so there is little to be gained from measuring them again. 
Third, there is also an ethical aspect. Is it acceptable to expose learners with a genuine professional 
development need to a learning environment that, judging from existing research, is very likely to be 
suboptimal? Our answer was “no.” Fourth and most importantly, exposing learners to a randomised 
experimental design would have implied informing them about the nature of the experiment, including 
the likelihood of participating in a control group. Providing this information is an ethical and legal 
obligation. We feared it would cause people to drop out once they realised they participated in the control 
group. This would have affected the very purpose of the investigation. 


286 





























The Effect of Multilingual Facilitation on Active Participation in MOOCs 
Colas, Sloep, and Garreta 

Analysis of Research Results 

Results of the HANDSON MOOC Experiment: Participation over Time 

Over the five-week duration of the experiment, MOOC participation dwindled, as was to be expected. 
Figure 2 describes the way in which participation unfolds during the five week period. Overall, out of 798 
participants who were active during week 1, 161 completed the first week’s module of activities. Among 
them were the 92 “designers” who eventually completed all the modules of the course and were awarded 
the designer badge and a course completion certificate as a result. The figure details four participant 
types: the designers and three kinds of non-designers. “Non-designers” either completed, started, or did 
not start at all the weekly modules of learning activities. 


For all language groups 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

92 

92 

92 

92 

92 

Non-designers** completing the module 

69 

23 

10 

7 

0 

Non-designers only starting the module 

637 

220 

64 

40 

31 

Non-designers not starting the module 

893 

1356 

1525 

1552 

1568 

Total (registered with a Canvas username) 

1691 

1691 

1691 

1691 

1691 


* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 
** Non-designers: all the other participants. 


All language groups 



Week 5 


400 


500 


600 


700 


800 


Number of participants (out of a total of 1691 registered; not starting the module not 

shown) 


Designers 
completing 
the module 
Non¬ 
designers 
completing 
the module 
Non¬ 
designers 
only 

starting the 
module 


Figure 2. Participation in weekly modules of activities (all language groups). 
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The pattern of activity showed a steady fall over the course duration, which was mostly due to the 
decrease in the number of non-designers, who only started the modules. Their number was 637 at the 
beginning of the course and went down to only 31 in week 5. The number of non-designers completing the 
module decreased similarly, so that by the end of week 5, designers represented the large majority of still 
active participants (92/123=74.8%). 


Differences between the seven language groups were observed in the way participation developed over the 
five weeks of the MOOC (Figures 3-9). 


For Bulgarian Group 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

26 

26 

26 

26 

26 

Non-designers** completing the module 

7 

2 

1 

0 

0 

Non-designers only starting the module 

76 

33 

9 

6 

2 

Non-designers not starting the module 

19 

67 

92 

96 

100 

Total (registered with a Canvas username) 

128 

128 

128 

128 

128 


* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 
** Non-designers: all the other participants. 


Bulgarian Group 

Designers 
completing 
the module 


Week 2 

Week 3 

Week 4 

Week5 

0 20 40 60 80 100 120 140 

Number of participants (out of a total of 128 having joined the bulgarian group; not starting the 

module not shown) 



I Non¬ 
designers 
completing 
the module 
I Non¬ 
designers 
only starting 
the module 


Week 1 



Figure 3. Bulgarian group: Participation in weekly modules of activities. 
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For Catalan Group 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

26 

26 

26 

26 

26 

Non-designers** completing the module 

22 

12 

5 

4 

0 

Non-designers only starting the module 

65 

25 

13 

5 

9 

Non-designers not starting the module 

27 

77 

96 

105 

105 

Total (registered with a Canvas username) 

140 

140 

140 

140 

140 


* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 
** Non-designers: all the other participants. 


Catalan Group 


Week 

Week 

Week 

Week 

Week 



120 


140 


Number of participants (out of a total of 140 having joined the Catalan group , not starting the 

module not shown) 


I Designers 
completing 
the module 
I Non¬ 
designers 
completing 
the module 
I Non¬ 
designers 
only 

starting the 
module 


Figure 4. Catalan group: Participation in weekly modules of activities. 
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For English Group 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

8 

8 

8 

8 

8 

Non-designers** completing the module 

14 

6 

3 

2 

0 

Non-designers only starting the module 

124 

44 

12 

6 

6 

Non-designers not starting the module 

64 

152 

187 

194 

196 

Total (registered with a Canvas username) 

210 

210 

210 

210 

210 

* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 

** Non-designers: all the other participants. 


English Group 

Week 1 

lplenng 

Week 2 

Week 3 

Week 4 

Week5 

the module 

Non- 

designers 

completing 

5' j the module 

Non- 

mh designers 

only 

starting the 
module 

20 40 60 80 100 120 140 

Number of participants (out of a total of 210 having joined the english group; not starting the 

module not shown) 


Figure 5. English group: Participation in weekly modules of activities. 
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For French Group 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

2 

2 

2 

2 

2 

Non-designers** completing the module 

5 

2 

1 

2 

0 

Non-designers only starting the module 

75 

25 

8 

2 

3 

Non-designers not starting the module 

39 

92 

110 

115 

116 

Total (registered with a Canvas username) 

121 

121 

121 

121 

121 

* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 

** Non-designers: all the other participants. 

F 

rench Group 



■■ Designers 
completing 

Week 1 I 





Week 2 1 

Week 3 1 

Week 4 1 

Week 5 1 

0 

me moouie 

tm Non¬ 
designers 
completing 
the module 

Non- 

■| designers 

only 

starting the 
module 

20 40 60 80 100 120 140 

Number of participants (out of a total of 121 having pined the french group; not starting the 

module not shown) 


Figure 6. French group: Participation in weekly modules of activities. 
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For Greek Group 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

28 

28 

28 

28 

28 

Non-designers** completing the module 

7 

0 

0 

0 

0 

Non-designers only starting the module 

40 

22 

3 

1 

1 

Non-designers not starting the module 

32 

57 

76 

78 

78 

Total (registered with a Canvas username) 

107 

107 

107 

107 

107 

* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 

** Non-designers: all the other participants. 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Greek Group 

!■ Designers 
completing 
the module 

^B Non¬ 
designers 
completing 
the module 

^B Non- 

designers 

only 

starting the 
module 

20 40 60 80 100 120 140 

Number of participants (out of a total of 107 having pined the greek group; not starting the 

module not shown) 


Figure 7. Greek group: Participation in weekly modules of activities. 
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For Slovenian Group 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

0 

0 

0 

0 

0 

Non-designers** completing the module 

0 

0 

0 

0 

0 

Non-designers only starting the module 

12 

6 

1 

1 

0 

Non-designers not starting the module 

7 

13 

18 

18 

19 

Total (registered with a Canvas username) 

19 

19 

19 

19 

19 

* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 

** Non-designers: all the other participants. 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

C 

Slovenian Group 

M Designers 
completing 
the module 

■■ Non- 

designers 
completing 
the module 

Non¬ 
designers 
only starting 
the module 

20 40 60 80 100 120 140 

Number of participants (out of a total of 19 having joined the Slovenian group , not starting the 

module not shown) 


Figure 8. Slovenian group: Participation in weekly modules of activities. 
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For Spanish Group 

Week 1 

Week 2 

Week 3 

Week 4 

Week 5 

Designers* completing the module 

2 

2 

2 

2 

2 

Non-designers** completing the module 

14 

1 

0 

0 

0 

Non-designers only starting the module 

92 

34 

9 

8 

4 

Non-designers not starting the module 

69 

140 

166 

167 

171 

Total (registered with a Canvas username) 

177 

177 

177 

177 

177 


* Designers: the 92 participants that completed all the 5 weekly modules and were awarded the Designer badge as a result. 
** Non-designers: all the other participants. 


Spanish Group 

Designers 
completing 
the module 
Non¬ 
designers 
completing 
the module 
Non¬ 
designers 
only starting 
the module 


0 20 40 60 80 100 120 140 

Number of participants (out of a total of 177 having joined the Spanish group, not starting the 

module not shorn) 


Week 1 

Week 2 

Week 3 

Week 4 

Week 5 



Figure g. Spanish group: Evolution of participation in weekly modules of activities. 

All seven groups showed the MOOC-wide decrease described above in the number of non-designers, but 
with some idiosyncrasies. Notably, the Greek non-designers only starting the module stood out from the 
rest of the groups, especially in week t, as they represented only 37% of the total number of participants of 
the Greek group, while this proportion ranged from 46 to 63% in the other groups. Also, of the 109 non¬ 
designers completing modules recorded during weeks 1-4 for all groups, 43 (39.4%) were from the 
Catalan group. The most salient difference between language groups was however in the number of 
designers, most of them (80/92=87%) belonging to only three language groups, the Bulgarian, Catalan 
and Greek ones. 

To refine the description of participants’ engagement and the comparisons between the language groups, 
as well as to gain further insights into the possible origins of observed differences, special attention was 
paid to completion behaviour following accomplishment of week 1. According to Laurillard (2015), activity 
in week 1 is a much better indicator of “intention to study” than “registration.” Indeed, 57% of the 
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participants who completed week l’s module completed the whole course, whereas only 10.2% of the total 
number of registrants did (Figure 10). 


Completion rates for HANDSON pilot 3 (by group; based on “Designer” and “Week 1” badges) 


Language 

Group 

(alphabetical 

order) 

Number 
of group 
members 
(M) 

Designer 

Badge 

(D) 

Completion 
rate of all 
group members 
= D/M 

Non-Designer 
Week 1 Badge 
(ND) 

Total 
Week 1 
Badge 
= D+ND 

Post-week 1 
Completion 
rate = 

D / [D+ND] 

Bulgarian 

128 

26 

20.3% 

7 

33 

78.8% 

Catalan 

140 

26 

18.6% 

22 

48 

54.2% 

English 

210 

8 

3.8% 

14 

22 

36.4% 

French 

121 

2 

1.7% 

5 

7 

28.6% 

Greek 

107 

28 

26.2% 

7 

35 

80.0% 

Slovenian 

19* 

0 

0 

0 

0 

o| 

Spanish 

177 

2 

1.1% 

14 

16 

12.5% 

Total 

902 

92 

10.2% 

69 

161** 

57.1% 


* Due to the low activity in the Slovenian group, after the first week it was decided to merge these participants with the English 
group. ** For their participation in the activities of Week 1,161 students obtained the Week 1 badge. 

D: All the owners of the Designer badge own the Week 1 badge. 

ND: Non-Designer W1 Badge: owners of the Week 1 but not the Designer badge. 
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Figure 10. Completion rates for the seven language groups. 
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The Greek group had the best post-week 1 completion rate. Of the 35 participants active in week 1 (having 
been awarded the week-i badge), 28 (80%) finished the course and obtained the “Designer Badge” 
(certificate of completion). This translated also into the best completion rate overall (26%), taking into 
account all the members of the Greek group. With similar rates, the Bulgarian group is second to the 
Greek one. The Catalan group with an overall 19% completion rate ranks third, although with only 54% of 
those who completed week 1 (also third in rank). No Slovenian participant obtained a badge. The 
completion rate of week-i badge owners for the three remaining groups, English (36%), French (29%) and 
Spanish (12%), was less than half of the Greek rate. 

Thus, large differences between language groups were observed in how the participation rates developed 
over the five weeks of the MOOC; activity during week 1 appeared to be a solid predictor of course 
completion. To gain insights into the possible causes for such differences, we turned to the participants’ 
evaluation of their experience with the multilingual facilitation. This information was obtained through 
surveys, especially those on multilinguality and facilitation (Figures 11 and 12, respectively), administered 
immediately after the course. 

Results of the HANDSON MOOC Experiment: User Survey 

The HANDSON MOOC participants who joined a language group were mostly from non-English speaking 
countries (692/902=76.7%; Figure 1). For 79% of the participants surveyed, the language group 
represented an environment conducive to their learning activities. Their command of English was quite 
high (not shown). Only 23% of the HANDSON MOOC participants were basic English users. 7% were 
native English speakers and the remaining 70% were intermediate level or proficient users. Only 11% of 
those who dropped out of the course did so because of insufficient English language knowledge (inferred 
from “the materials were in English and I did not expect that”). For comparison, the most common reason 
for deciding not to continue the MOOC was lack of time (for 67% of those dropping out; not shown). 

Even though reading materials in English did not appear to present a major obstacle to participation and 
in spite of various announcements to the contraiy, a small majority of the participants expected to find all 
materials translated in their own language (51%) and would certainly have preferred it in their own 
language (54%) (Figure 11). Accordingly, only 49% found the weekly Hangout live session of 
videoconference in English easy to follow, whereas the weekly Hangouts in their own or preferred 
language were better received, and reported as useful for 67%. Despite these notable difficulties with 
listening to Hangouts in English, in the experience of 65% there was a good balance between the materials 
and sessions in English and the language groups. 
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Agree or Strongly agree 


Neutral 


Disagree or Strongly disagree 


(%) 


The weekly Hangouts in my preferred language 
were useful 

The weekly Hangouts in Engish were easy to 
foiow 

The group spaces per language helped create a 
sense of community 

The daiy course journal On your language group) 
was a good motivation tool 

The course announcements On English) helped to 
keep informed about the MOOC 

The balance between the materials and sessions 
in Engish and the language groups was good 

I would have preferred everything in my 
language 

I think that having facitators in my own language 
is very helpful 

I enjoyed being able to participate and contribute 
in my own language 

I expected to find al materials translated 

0 


67.1 


26.8 


m 


48.8 


37.8 


73.2 


63.4 


68.3 


64.7 


53.6 


20.7 


31.7 


s 

s 


28 


79 3 14.6 


78 13.4 


25 


51.2 


50 


24.4 


75 


10U 


Figure n . Results of the survey of the participants about multilinguality. 


Overall then, although they had a good command of English and the English MOOC reading materials did 
not present a major obstacle to them, a large majority (76%; not shown) enjoyed being part of a 
multilingual MOOC. 78% enjoyed contributing in their own language and 79% found it very helpful 
having a facilitator in their own language. Responses they received from the facilitators were found 
helpful (82%). Instructions provided by facilitators to the whole group were clear (78%; Figure 12). 
Facilitators adequately helped to cope with the problems experienced during the learning activities (76%). 
Overall, respondents were very satisfied with facilitators, their role, help, guide and motivation. In 
accordance with this positive assessment, only 2% of the participants who dropped out of the MOOC did 
so because they felt the facilitators did not provide enough guidance (not shown); this reason for dropping 
out is the least important of all the reasons recorded for the HANDSON MOOC. 
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Agree or Strongly agree ■■ Neutral Disagree or Strongly disagree 


The introduction to the course (initial video, initial 
instructions, etc.) was a good preparation for 
the rest of the activities of the course 

FadStators adequately helped me to cope with 
the problems I experienced during the learning 
activities 

The facitation provided during the course was 
insufficient to guide me with the activities 

There were too many different facirtators 


The instructions provided by facirtators to the 
whole group were dear to me 

The responses from the facirtators were helpful 


I would have Bred a more active role from 
facirtators 


The facirtators helped me keep engaged 

0 



Figure 12. Results of the survey of the participants about facilitation. 


Thus, in the participants’ feedback on the multilingual facilitation no factor stood out as a possible cause 
for observed engagement differences between language groups. The fact that the English group ranked 
only fourth in completion rates also seems to confirm that multilingual facilitation performed well enough 
to compensate for the disadvantage of non-native learning materials, and was thus largely irrelevant for 
explaining differences in completion behaviours between language groups. 

This apparent explanatory irrelevance of the act of facilitation and our inability to find any obvious clues 
corresponding to observed behaviour differences, prompted us to search for other possible predictors of 
these different behaviours. Since participant engagement seems to have settled already at the end of the 
first week, we looked for still earlier determinants which could be identified already in the very first days 
of the MOOC. We identified two instruments for probing the differential activity of the seven language 
groups that met our criteria: Mailchimp and Canvas Groups Pageviews. 
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A Mailchimp campaign was launched on the second day of the MOOC already. It consisted of a welcoming 
message to the participants and contained a link to further information. It was sent separately to each 
language group and was phrased in the group’s language. The result of this campaign (“open and click 
rates”) were extracted two days later (Figure 13). 



Response to first Mailchimp message (Oct 28-30, 2014) 

Language group 
(alphabetical order) 

Open rate 

% | Ranking 

Click rate 

% | Ranking 

Open + Click 

% | Ranking 

Bulgarian 

61.8 

3rd 

43.7 

1st 

105.5 

2nd 

Catalan 

80.3 

1st 

42.0 

2nd 

122.3 

1st 

English 

44.4 

6th 

14.9 

6th 

59.3 

6th 

French 

49.2 

5th 

18.9 

4th 

68.1 

4th 

Greek 

70.8 

2nd 

28.7 

3rd 

99.5 

3rd 

Slovenian 

32.1 

7th 

10.7 

7th 

42.8 

7th 

Spanish 

49.7 

4th 

16.6 

5th 

66.3 

5th 

Average 

55.5 

/ 

25.1 

/ 

80.5 

/ 



Figure 13. Response to early Mailchimp campaign by the seven language groups. 
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Three language groups ranked above average in their response to the Mailchimp message: the Catalan, 
Bulgarian and Greek ones. Response rates of the four remaining groups were below average. Thus, already 
on the second day in the MOOC’s history, the differences in engagement between the language groups 
match those that were observed later on, throughout the remaining runtime of the course. The use of the 
language group pages in Canvas complement these findings. Use of the MOOC Canvas platform by 
participants was evaluated through the analysis of the number of times the group pages were viewed (the 
group page was accessed each time participants engaged in the learning activities of their group). 
GoogleAnalytics Pageview was used to extract this information during the five weeks the MOOC ran, as 
well as the week before and after that (Figure 14). 



- Bulgarian 

— Catalan 

- Greek 

- Slovenian 

- English 

- French 

- Spanish 

—— Average of all 
groups 


Figure 14. Group Pageviews. 


As may be observed in “Average of all groups” (black line), page viewing reached a maximum during week 
1, and then decreased steadily during the following weeks. The Catalan group showed a trend very similar 
to the average of all groups. The Bulgarian and Greek groups stood out with a first week’s peak of page 
viewing roughly double the average value. In contrast, page viewing for the remaining four groups 
(English, French, Slovenian, and Spanish) was below average during the whole duration of the MOOC. 
For week-i page viewing the seven groups ranked as follows: 1st Bulgarian, 2nd Greek, 3rd Catalan, 4th 
French, 5th Spanish, 6th English and 7th Slovenian. This ranking was maintained over the entire 
measurement period. 
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Thus, information from page viewing was in general agreement with data on module completion, 
especially of week-i completion, and represented a good indicator of engagement in learning activities. 
Page viewing, it seems, gives valuable information on participants’ daily engagement behaviours, even 
before completion of week 1 - that is, during the very first days of the MOOC. This behaviour was 
therefore detailed during a period of n days starting from the day of the kick-off hangouts video- 
conference, preceding by three days the official day 1 of the MOOC up to the first day of week 2 (Figure 
15)- 
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- Bulgarian 
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- Slovenian 

- English 

- French 

- Spanish 

—**— Average of all 
groups 


Figure 15. Group Pageviews during the first days. 


The ordering previously observed for engagement in week t’s learning activities was conserved during the 
entire MOOC: an average or above average cluster consisting of the Bulgarian, Greek and Catalan groups 
and a below average cluster of English, French, Slovenian and Spanish groups. This order was already in 
place during the very first days of the MOOC, even on day 1. For the Greek group, group page viewing had 
started even the day before, on October 26th, before any official facilitation action had occurred. 


Table 1 summarises the measurements recorded on participation throughout the MOOC, from day 1 up to 
course completion (owing to newly identified early indicators of engagement: Pageviews and Mailchimp). 
In addition, each of the seven language groups is given a colour code to highlight its ranking for each 
MOOC event. This illustrates at a glance one of our main findings: the top-three language groups in terms 
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of completion rates (Bulgarian, Catalan and Greek), were already ranked in the three top-most positions 
in terms of engagement from the first days of the course, that is, before multilingual facilitation could 
have had any impact. 


Table 1 

Ranking (#1-7) and Possible Early Predictors of Course Completion for the Seven Language Groups 


Ranking of 
language 
groups for 
each MOOC 

events 

Day 1 

Oct 27, 2014 

Early Week 1 

Oct 28-30, 2014 

Week 1 

Oct 27-Nov 2, 2014 

After Week 1 

NOV 3-28, 2014 

Course 

Pageviews 

Mailchimp 
Open and click 

rate 

Course 

Pageviews 

Week 1 Badges 
(% all members) 

Completion 
rate of Week 1 
badge owners 

Completion rate 
of all group 
members 

#1 

1.46 


9.90 


80.0% 

26.2% 

#2 

1.04 

105.5 

8.82 

35 (32.7%) 

78.8% 

20.3% 

#3 


99.5 


33 (25.8%) 



#4 

0.68 

68.1 

3.60 

22 (10.5%) 

36.4% 

3.8% 

#5 

0.63 

66.3 

3.02 

16 (9.0%) 

28.6% 

1.7% 

#6 

0.36 

59.3 

2.44 

7 (5.8%) 

12.5% 

1.1% 

#7 

0.26 

42.8 

1.47 

0 (0%) 

/ 

/ 

Average 

0.75 

80.5 

4.73 

23 (17.8%) 

57.1% 

10 .2% 


Group color 

English 

Spanish 


Bulgarian 

French 

Greek 

Slovenian 

Number of 

members 

210 

177 

140 

128 

121 

107 

19 


Conclusions and Discussion 

The HANDSON MOOC experiment sought to investigate the veracity of two hypotheses: i) whether using 
English only as the language of instruction negatively affects the completion rate (Hypothesis 1) and ii) 
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whether using each participant’s native (or preferred) language for facilitation (both by peers and by a 
dedicated facilitator) would boost participation (Hypothesis 2). We’ll now confront these hypotheses in 
turn with the results obtained. 

Hypothesis 1 can best be evaluated by comparing the HANDSON MOOC’s completion rates with those 
obtained in the ICT in Primary Education MOOC (IPE; Laurillard, 2015) as this was a course for teacher 
training with instruction in English too. In agreement with the findings of Jordan (2014) who found that 
“approximately 50% of MOOC students who sign up go on to become active users,” 55% of those who had 
registered at the start of the IPE course were active during the first week. With 54%, this figure was 
similar for HANDSON. Moreover, in both MOOCs, approximately 10% of those active in the first week 
received a certificate of course completion. Third, overall completion rates were almost identical (IPE 
5.3% vs. HANDSON 5.4%). Looking at data on completion for other courses as compiled by Jordan 
(2014), at that time a 5% completion rate was about the norm. We conclude therefore that our findings 
confirm our first hypothesis: using English as the language of instruction does not negatively affect 
completion for an audience for which English is only one of the native languages present. 

However, behind the overall rate of 5.4% quite a bit of diversity is hidden. Three groups (Bulgarian 10.8%, 
Catalan 9.9% and Greek 14.0%) outperformed the IPE reference whereas the other four groups (English 
2.0%, French 0.9%, Slovenian 0% and Spanish 0.6%) underperformed with respect to IPE. Note that all 
these groups were facilitated in the participants’ preferred (native) language. It seems that, unlike what 
Hypothesis 2 assumes, multilingual facilitation in some cases doubles or triples completion but in other 
cases more than halved it. Hypothesis 2, therefore cannot be confirmed, at least not in its present form. 
This of course prompts the question why this is so. In our discussion below we will discuss how both 
groups of languages differed and we will suggest two additional hypotheses - that our data suggest but 
obviously go untested - for how one may account for the observed difference between them. 

Threshold Group-size Values 

Our findings suggest that it is impossible to change the course of events for learner engagement after week 
1, or indeed, even after the first day. The strict separation in rankings of the high-completion cluster of the 
Bulgarian, Catalan and Greek groups on the one hand and the low-completion cluster of English, French, 
Slovenian and Spanish groups on the other hand remains intact throughout the five weeks the MOOC 
lasted. This suggests, we surmise, the existence of early threshold values that need to be met in order to 
ensure subsequent engagement in learning activities. 

Of primary importance seems to be the size of the group of fully active participants during week 1, as 
measured by the number of week-i badges awarded. Figure 10 shows there is a correlation (Spearman’s 
rho equals 0.927) between final completion rates (“designer badge”) and the numbers of participants 
completing week 1 (“total week 1 badge”) (p=.003). Note that no such correlation can be established 
between final completion rates and the initial size of the group (Spearman’s rho .083, p=.86o). Indeed, 
the Greek group had the best post-week-i completion rate (80%). This rate was lower in groups that were 
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initially larger than the Greek one: English (36%), French (29%) and Spanish (12%). We suggest that the 
latter groups’ low completion rates are due to the low numbers of active participants in week 1 (22, 7 and 
16 respectively, while the Bulgarian, Catalan and Greek groups had 33, 48 and 35 active participants, 
respectively). One may surmise that such low numbers are insufficient to reach a critical threshold 
number necessary for good operation of the peer-tutoring activities (especially critical during week 1), for 
building a sense of community, and for ensuring engagement through facilitation of learners for the 
remaining four weeks (threshold hypothesis ). 

A threshold of 25-30 fully active participants appears a good approximation for the minimal number 
required to ensure engagement past the first week. This was only obtained in three groups (the Bulgarian, 
Catalan and Greek groups, cf. Table 1). Interestingly, in the typology of social systems for supporting 
learning (Dron & Anderson, 2014), 30 persons is the size delimiting a group from a network. Moreover, 
the emergence of a network-like structure is critical for the development of learning support in large 
online courses such as cMOOCs (Sloep, 2009; Downes, 2013). Thus staying below the size of 30 is likely to 
have prevented the low-completion language groups (English, French and Spanish) from adapting to the 
relatively unstructured learning environment of the HANDSON MOOC. 

Furthermore, we suggest that threshold values separating the two clusters of high-completing and low- 
completing groups not only refer to the numbers of week-i completers but also include other values: 
number of group Pageviews and responses to Mailchimp. As these apply even before week-i completion, 
there seems to be a chronologically ordered series of predictive threshold values. This extension of the 
threshold hypothesis predicts that for obtaining at least the 10% completion rate observed on average for 
the seven groups that actually started the HANDSON MOOC, one would not only need to have at least 30 
participants completing week 1 in a group (factor 1) but also, even before that, a Mailchimp (or similar) 
response rate of at least 80% (factor 2), or an estimated number of course Pageviews of at least 4.0 for 
week 1 or at least 0.75 for day 1 (factor 3). (The figure of 10% corresponds to 92/902=10.2%, cf. Figure 10; 
if one wants to compare with other MOOCs, this corresponds to 5.4% if all 1691 registered participants are 
taken into account.) It would seem therefore that only slight differences early on can have a considerable 
impact on final completion rates: compare for example the Catalan and Spanish groups on either side of 
the threshold of 0.75 for day-i page viewing, with 0.79 and 0.68 respectively, and ending up with a 
completion rate of 18.6% and 1.1% respectively (Table 1). Most importantly, even though the initial size of 
a group (factor 1) alone does not give reliable indication on the group’s future success, these indicators 
and thresholds (factors 2 and 3) seem to be strongly suggestive - as early as from day 1 - of the capacity of 
a group to organise successfully as a learning network of active participants. 

The apparently required initial size of a group has some theoretical grounding in the work of Dunbar, that 
is, in the so-called Dunbar number (Dunbar, 1993; Hill & Dunbar, 2002). This number is used to argue 
that a course needs to attain about 150 active participants in order to be considered “massive,” that is, to 
enable a “connectivist” dynamic (Downes, 2009, 2013; Dron & Anderson, 2014). The average group size 
in the HANDSON MOOC of 147.2 seems to qualify. Our findings suggest, however, that a group does not 
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necessarily need 150 active participants to operate as a social learning network. 30 active participants 
already suffice. Groups in which fewer than 20% of those who signed up were converted into active 
participants during the first week ended up in the low completion cluster, regardless of whether the initial 
size of the group was bigger or smaller than Dunbar’s number of 150. The groups of the high completion 
cluster all counted fewer participants than Dunbar’s number of 150 and still reached at least 30 active 
participants thanks to a first-week conversion rate substantially higher than 20%. These observations 
seem to weaken the theoretical significance of Dunbar’s number. However, Hill and Dunbar (2002) 
discern five different levels of group size, one of them being 21. That number is much closer to our finding 
of 30 active participants. Indeed, Sloep (2015) argues that 150 is a maximum. Above it, and probably well 
below it already, one needs supportive structures, such as the already mentioned ad-hoc transient 
communities (“pop-up communities”), to kick off and maintain a sustainable social dynamic in a learning 
network. It is tempting, then, to speculate that “inactive participants” are somehow also important for the 
emergence of a group of 30 active participants. This ties in with work that underscores the importance of 
lurkers (Preece, Nonneke, & Andrews, 2004) as well as with recent findings on the function of inactivity in 
decentralised complex systems (Charbonneau & Dornhaus, 2015). Therefore the conversion rate 
threshold of 20% is likely to be meaningful and might be used, together with the absolute number of 
active participants, as a reliable indicator of sustainable social dynamics during the first week of a MOOC. 

Community Factors 

Our discussion thus far suggests an explanation for the success of the Bulgarian, Catalan and Greek 
groups. But why then failed the other groups, including the initially two largest ones, to engage in the first 
week’s activities? The Greek facilitators may have been very active and significantly more supportive at 
the start of the course than did the facilitators in other language groups. However, this could hardly have 
had an impact that explains the lead already taken by the Greek group on the day previous to the start of 
the MOOC, when facilitation had not officially started yet. Moreover participants surveyed only expressed 
an overall satisfaction with multilingual facilitation and the group space per language was found to help 
create a sense of community by 73% of them. Thus, differences in facilitation do not appear to explain the 
differences in engagement between language groups. 

A more likely hypothesis for interpreting the low completion rates observed in the two initially largest, 
English and Spanish groups invokes the notion of a pre-existing sense of community. Indeed, in contrast 
to the members of the best performing Bulgarian, Catalan and Greek groups, who share a strong cultural 
identity, these two large groups included participants from all over the world. Other than sharing a 
common language they had little in common culturally speaking (this also holds true for the smaller 
French group). This absence of strong cultural ties and perhaps also of geographical proximity among 
participants and facilitators may have made the creation of a sense of community much harder, as well as 
the promotion of peer tutoring and participation in learning activities. Interestingly, out of the six cultural 
dimensions (Power distance, Individualism, Masculinity, Uncertainty avoidance, Long term orientation, 
and Indulgence) used to compare various countries (Nkuyubwatsi, 2014), the only dimension for which a 
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clustering is reproduced similar to that for course completion of language groups is ‘Individualism’ (not 
shown). The fact that a low level of individualism corresponds to the high-completion cluster of language 
groups in the MOOC fits well with the notion of a strong, pre-existing sense of community for these 
groups. 

Along with this purely cultural aspect, the participants of the Bulgarian, Catalan and Greek groups may 
have also benefited from having been recruited to the MOOC from a common trusted source (as opposed 
to, for example, impersonal advertisements in MOOC lists for the other four groups). This was especially 
the case for the participants of the Greek group, who were recruited through the HANDSON partner 
Ellinogermaniki Agogi. They sent out invitation emails through the e-twinning mailing list to all primary 
and secondaiy Greek Schools. A third example of pre-existing connections comes from the Catalan 
participants, who were linked with each other through a common incentive: completing at least 80% of 
the learning activities gained them an official title from the Generalitat of Catalunya. 

These three factors all point into the direction of a rational decision making process on the part of the 
participants: before investing time in a MOOC, one wants to make sure that the time is well spent. Lack of 
space forbids us to explore this community perspective in further depths (but see Kester et al., 2007). 
Jointly, we will call these three explanations the tried and trusted community hypothesis. 

In summary, our discussion points out that for multilingual facilitation to be effective two more 
conditions need to be fulfilled: an above threshold group size and a tried and trusted community. They 
appear critical for the formation of the learning network that constitutes the basis for the social dynamics 
of active participation in MOOCs. Further research is needed to investigate the general validity of this 
claim and to refine our understanding of how the factors of multilingual facilitation, a threshold group 
size and pre-existing ties within the group act in concert. Regardless, to the extent that MOOCs are the 
new form of distance education, which goes in particular for the MOOCs emerging in Europe (Deimann & 
Vogt, 2015), our findings offer insights that may profitably be used both to design new distance learning 
offerings and to give distance education theory new inputs that still align with its original tenets and 
intentions of serving underprivileged and culturally heterogeneous audiences (Lundahl, 2014; Sloep & 
Schuwer, 2015). 
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HANDS-ON ICT - PostPilot sirvey Hands on ICT for Riot 3 


About the course and its approach 


How much do you agree about facilitation statements? * 






Please choose the appropriate response for each item; 

1 

2 

3 

4 

5 

The introduction to the course (initial video, initial instructions, 
etc.) was a good preparation for the rest of the activities of the 

O 

o 

o 

o 

o 

course 






Facilitators adequately helped me to cope with the problems 1 
experienced during the learning activities 

O 

o 

o 

o 

o 

The facilitation provided during the course was insufficient to 
guide me with the activities 

O 

o 

o 

o 

o 

There were too many different facilitators 

O 

o 

o 

o 

o 

The instructions provided by facilitators to the whole group were 
clear to me 

O 

o 

o 

o 

o 

The responses from the facilitators were helpful 

o 

o 

o 

o 

o 

1 would have liked a more active role from facilitators 

o 

o 

o 

o 

o 

The facilitators helped me keep engaged 

o 

o 

o 

o 

o 

(1= strongly disagree .5= strongly agree) 







How much do you agree about the multilinguism in the MOOC statements? * 





Please choose the appropriate response for each item 

1 

2 

3 

4 

5 

The weekly Hangouts in my preferred language were useful 

O 

o 

o 

o 

o 

The weekly Hangouts in English were easy to follow 

O 

o 

o 

o 

o 

The group spaces per language helped create a sense of 
community 

0 

o 

0 

0 

o 

The daily course journal (in your language group) was a good 
motivation tool 

O 

o 

o 

o 

o 

The course announcements (in English) helped to keep informed 
about the MOOC 

O 

o 

o 

o 

o 

The balance between the materials and sessions in English and 
the language groups was good 

O 

o 

o 

o 

o 

1 would have preferred everything in my language 

O 

o 

o 

o 

o 

1 think that having facilitators in my own language is very helpful 

O 

o 

o 

o 

o 

1 enjoyed being able to participate and contribute in my own 
language 

O 

o 

o 

o 

o 

1 expected to find all materials translated 

O 

o 

o 

o 

o 

(1= strongly disagree .5= strongly agree) 







httpy/sirvey.e(Tura.si/index.ph(i/admirV|pnntabiesurvey/sa/index/surveyid/513967 
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Table PostPilot3 - Which Language Group Did You Join. 


value 

N 

raw% 

valid % 

cumulative % 

Bulgarian 

23 

27.71 

28.05 

28.05 

Greek 

18 

21.69 

21.95 

50.00 

Catalan 

17 

20.48 

20.73 

70.73 

English 

12 

14.46 

14.63 

85.37 

Spanish 

7 

8.43 

8.54 

93.90 

French 

5 

6.02 

6.10 

100.00 

missings 

1 

1.20 




total N=83 ■ 

valid N=82 

x=2 9 9 • o=175 


Table Pre-Course Activities - Facilitators Training. 


HANDSON pilot 3 / Pre-MOOC training period for facilitators (Aug. 27th - Oct. 24th, 

2014) 


• 27th August: Kick-off facilitators meeting - 3 pm CEST 

• 5th September: Open session to ask questions and “play” with Hangouts - 3 pm CEST 

• 19th September: Introduction to the HANDSON MOOC tools (Hangouts) - 3 pm CEST 

• 26th September: “Meet the facilitators” video recording (Hangouts) - 3 pm CEST 

• 3rd October: Facilitators training (Hangouts) - 3 pm CEST 

• lord October: Facilitators training (Hangouts) - 3 pm CEST 

• I7rd October: Facilitators training (Hangouts) - 3 pm CEST 

• 24th October: MOOC kick-off (Hangouts) - 3 pm CEST 


0 Athabasca 
University 
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